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(54) Packet multiplexing system 

(57) To provide a simply-configured multiplexing 
system which detects multiplexing errors, the PES pack- 
etizing modules 11, 21, and 31 packetize elementary 
streams (ES) to produce packetized elementary 
streams (PES), and the TS/PS packetizing module 41 
multiplexes packetized elementary streams (PES) to 
produce transport streams (TS) or program streams 



(PS). The comparators 1 3, 23, 33, 45, 46, and 47 each 
compare an non-packetized elementary stream (ES) or 
a packetized elementary stream (PES) stored in the 
FIFO memory with the packetized elementary stream 
(ES) or packetized elementary stream (PES) from the 
PES packetizing module 11, 21, 31, and 41. Based on 
the comparison result, the control module 5 detects an 
error in the PES packetizing module 11,21, 31, and 41. 
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Description 

BACKGROUND OF THE INVENTION 

1 , Field of the invention 

The present invention relates to a packet multiplex- 
ing system which divides input data into a plurality of 
packets to multiplex the packets for output. The inven- 
tion relates particularly to a packet multiplexing system 
which compares input data with the corresponding data 
in a packet. 

2. Description of the prior art 

MPEG2 (Moving Picture Expert Group 2), one of en- 
coding systems for compressing moving pictures, has 
been used to generate packets of various types of data 
such as image data, voice data, text data, and so forth. 
These packets are multiplexed, recorded on recording 
media, and sent over communication lines. MPEG2 de- 
fines the recommendations for a unit receiving multi- 
plexed packets that is a de-multiplexing system, but not 
for a multiplexing system. Therefore, a multiplexing sys- 
tem may be designed as long as a de-multiplexing sys- 
tem satisfies the recommended performance. 

A conventional packet multiplexing system has the 
video PES packetizing module 61 , voice PES packetiz- 
ing module 62, private data PES packetizing module 63, 
and TS/PS packetizing module 64 as shown in Fig. 2. 
The three packetizing modules, video PES packetizing 
module 61, voice PES packetizing module 62, and pri- 
vate data PES packetizing module 63, packetize ele- 
mentary streams (ES), such as video data, voice data, 
and private data, respectively, which are sent from the 
encoding module (not shown). The packetizing modules 
61 , 62, and 63 then generate packetized elementary 
streams (PES) shown in Fig. 3. Next, the TS/PS pack- 
etizing module 64, in turn, packetizes those PESs, gen- 
erate transport streams (TS) or program streams (PS) 
shown in Figs. 4 and 5, and outputs them. 

In addition, this packet multiplexing system com- 
pares non-multiplexed data and multiplexed data in the 
stream to check if input data is set up correctly in the 
stream. More specifically, the packet multiplexing sys- 
tem has a circuit (not shown) functionally equivalent to 
the packet de-multiplexing module which should be in a 
de-multiplexing system. This circuit, almost as large as 
the multiplexing system in size, de-multiplexes the mul- 
tiplexed transport streams (TS) or program streams 
(PS) into elementary streams (ES); that is, video data, 
voice data, and private data elementary streams (ES), 
and then checks if the de-multiplexed elementary 
streams (ES) match the original elementary streams 
(ES). 

However, the monitor circuit in the conventional 
packet multiplexing system is too large and expensive, 
which leads to difficulties to provide a compact and low- 
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cost packet multiplexing system. 
SUMMARY OF THE INVENTION 

s To solve the above problem, a packet multiplexing 
system according to the present invention generates, 
based on input data, a packet containing the input data 
and a header, and compares the data in the generated 
packet and the input data. This allows determination of 

>o whether input data matches data in a generated packet. 
Therefore, the packet multiplexing system according to 
this invention is compact and low-cost as compared with 
the conventional packet multiplexing system. 

'5 BRIEF DESCRIPTION OF THE DRAWINGS 

The present invention will be understood more fully 
from the detailed description and the accompanying 
drawings below of the preferred embodiments of the in- 
20 vention. 

In the drawings: 

Fig. 1 is a block diagram showing the configuration 
of a multiplexing system according to the first em- 
2$ bodiment of this invention; 

Fig. 2 is a block diagram showing the configuration 
of a conventional multiplexing system; 
Fig. 3 is a diagram showing the format of a pack- 
etized elementary streams (PES); 
30 Fig. 4 is a diagram showing the configuration of a 
transport stream (TS); 

Fig. 5 is a diagram showing the configuration of a 
program stream (PS); and 
Fig. 6 is a block diagram showing the configuration 
35 of a multiplexing system according to the second 
embodiment of this invention. 

DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

The multiplexing system according to the present 
invention may be employed to the multiplexing module 
of an encoder which multiplexes image data, voice data, 
and private data compression-encoded using a com- 
45 pression technology such as MPEG2. 

<First Embodiment 

In the first embodiment, the present invention is ap- 
se plied to the multiplexing module of such an encoder. As 
shown in Fig. 1 , this multiplexing module includes three 
packetizing systems which packetize data sent from the 
encoding module of the encoder. Three packetizing sys- 
tems are: the video packetized elementary stream 
55 (PES) packetizing system 10 which packetizes video 
data, voice PES packetizing system 20 which packetiz- 
es voice data, and private data PES packetizing system 
30 which packetizes private data. In addition, the multi- 
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plexing module has the TS/PS packetizing system 40 
which multiplexes the packetized elementary streams 
sent from the PES packetizing systems 10, 20, and 30. 

The video PES packetizing system 1 0 has the video 
PES packetizing module 1 1 to packetize the video data 
elementary stream ES, the First In First Out (FIFO) 
memory 1 2 to store non-packetized video data, and the 
comparator 1 3 to compare non-packetized video data 
with packetized video data. The voice PES packetizing 
system 20 and the private data PES packetizing system 
30 each have the same configuration as that of the video 
PES packetizing system 10. More definitely, the voice 
PES packetizing system 20 and the private data PES 
packetizing system 30 have the voice PES packetizing 
module 21 to packetize voice data elementary streams 
and the private data PES packetizing module 31 to 
packetize private data elementary streams, respective- 
ly, the FIFO memory 22 to store non-packetized voice 
data and the FIFO memory 32 to store non-packetized 
private data, respectively, and the comparator 23 to 
compare non-packetized voice data and packetized 
voice data and the comparator 33 to compare non-pack- 
etized private data and packetized private data. 

Each of the PES packetizing modules 11, 12, and 
13 adds a header and other information to an elemen- 
tary stream (ES) sent from each encoding module to 
form a packetized elementary stream (PES) shown in 
Fig 3. The length of a packetized elementary stream 
(PES) is variable. 

The PES packetizing module 11, 21, or 31 requires 
some time to process an elementary stream (ES) to form 
a packetized elementary stream (PES). This results in 
a time delay between time when an elementary stream 
(ES) is sent to the packetizing module 11 , 21 , or 31 and 
time when a packetized elementary stream (PES) is pro- 
duced. To compare a packetized elementary stream 
(PES) with an elementary stream (ES), the payload field 
of the packetized elementary stream (PES), shown in 
Fig. 3, and the elementary stream (ES) must be simul- 
taneously sent to the comparator 1 3, 23 or 33, respec- 
tively. In short, each of FIFO memories, 12, 22, or 32, 
need only be large enough to contain an elementary 
stream (ES) that is sent within the delay time of the PES 
packetizing modules 11 , 21 , or 31 ; if the input data rate 
is "V" and the delay time is "T", then the FIFO memory 
size "M" need only be at least "V° x "T". 

To synchronize a packetized elementary stream 
(PES) with an elementary stream (ES), the FIFO mem- 
ory delays the elementary stream (ES) by a predeter- 
mined time (e.g., the sum of the time required to create 
the header and the time to create the packetized ele- 
mentary stream (PES)). However, the time required to 
generate a packetized elementary stream varies from 
time to time. So, it is desirable that, instead of predeter- 
mining a time for the FIFO memory, the packetizing 
module monitors the generation of a packetized ele- 
mentary stream and sends an elementary stream to the 
FIFO memory when the packetized elementary stream 



is generated. 

The TS/PS packetizing system 40 has the TS/PS 
packetizing module 41 to multiplex packetized elemen- 
tary streams (PES) sent from the PES packetizing mod- 

s ules 11, 21, and 31 to form a transport stream (TS) 
shown in Fig. 4 or a program stream (PS) shown in Fig. 
5, the FIFO memories 42, 43, and 44 to contain non- 
multiplexed packetized elementary streams (PES) from 
the above-mentioned PES packetizing modules 11, 21 , 

10 and 31 , and the comparators 45, 46, and 47 to compare 
the non-multiplexed packetized elementary streams 
(PES) and the packetized elementary streams (PES) 
multiplexed by TS/PS packetizing module 41 . 

A transport stream (TS) contains one or more pack- 
's etized elementary streams (PES) divided intodata items 
each of a specified length. Thus, depending upon the 
length of a packetized elementary stream (PES), a plu- 
rality of packetized elementary streams (PES) form one 
transport stream (TS) or, conversely, one packetized el- 

zo ementary stream (PES) extends across a plurality of 
transport streams (TS). A program stream (PS), which 
is created by adding a header to a packetized elemen- 
tary stream (PES), is never divided. 

Because multiplexing requires some time, there is 

25 a time delay between time when a packetized elemen- 
tary stream (PES) is sent to the TS/PS packetizing mod- 
ule 41 and time when a transport stream (TS) or a pro- 
gram stream (PS) is output. Therefore, as with the FIFO 
memories 12, 22, and 32, the FIFO memories 42, 43, 

30 and 44 each must be large enough to hold packetized 
elementary streams (PES) that are output during the de- 
lay time in order to send them to the comparators 45, 
46, and 47 at the same time a transport stream (TS) or 
a program stream (PS) is sent from the TS/PS packetiz- 

35 ing module 41 to the comparators 45, 46, and 47. 

In the multiplexing module with a configuration de- 
scribed above, the elementary streams (ES) of video da- 
ta, voice data, and private data are sent from the encod- 
ing module to the video PES packetizing module 11, 

40 voice PES packetizing module 21 , and private data PES 
packetizing module 31 before being packetized. The el- 
ementary streams (ES) are also sent to the FIFO mem- 
ories 12, 22, and 32. 

The video PES packetizing module 11 addsahead- 

45 er and some other information to the elementary stream 
(ES) of the received image data to form a packetized 
elementary stream (PES). It then sends this packetized 
elementary stream (PES) to the comparator 1 3 and the 
FIFO memory 42. 

so From the packetized elementary stream (PES) sent 
from the video PES packetizing module 11 , the compa- 
rator 13 extracts the part corresponding to the elemen- 
tary stream (ES) and compares it with the elementary 
stream (ES) before packetizing sent from the FIFO 

ss memory 12. If there is no error in the packetized ele- 
mentary stream (PES) generated by the video PES 
packetizing module 11, the non-packetized elementary 
stream (ES) should match the packetized elementary 
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stream (ES); if there is an error, the non-packetized el- 
ementary stream (ES) should not match the packetized 
elementary stream (ES). In the latter case, the compa- 
rator 13 informs the control module of the condition. 

The other two packetizing modules, the voice PES 
packetizing module 21 and the private data PES pack- 
etizing module 31 , do the same processing, More spe- 
cifically, they add a header to a received elementary 
trea 1 jrm a pack nized elementary stream 
(PES) and sends it to the comparator 23 and the FIFO 
memory 43 , or to the comparator 33 and the FIFO mem- 
ory 44, respectively. 

From the packetized elementary stream (PES) sent 
from the video PES packetizing module 21 or 31, the 
comparator 23 or 33 extracts the part corresponding to 
the elementary stream (ES) and compares it with the 
non-packetized elementary stream (ES) sent from the 
FIFO memory 22 or 32 and informs the control module 
5 of the result. 

In this manner, the control module 5 receives the 
result of comparison between the non-packetized ele- 
mentary stream (ES) and the packetized elementary 
allows the control module 5 to check 
whether or not an error was generated during execution 
of the video PES packetizing modules 11, 12, or 13. 

A packetized elementary stream (PES) from the 
PES packetizing module 11 , 21 , or 31 is sent to the TS/ 
PS packetizing module 41 . The TS/PS packetizing mod- 
ule 41 multiplexes a packetized elementary stream 
(PES) from each of the video PES packetizing modules 
11,21, and 31 to form a transport stream (TS) or a pro- 
gram stream (PS), and sends this transport stream (TS) 
to the communication line and to each of the compara- 
tors 45, 46, and 47. 

From a transport stream (TS) or a program stream 
(PS), the comparator 45 extracts the part corresponding 
to the packetized elementary stream (PES) of video da- 
ta and compares it with the non-multiplexed packetized 
elementary stream (PES) stored in the FIFO memory 42 
and informs the control module of the comparison result. 
When the comparator 45 receives a transport stream 
(TS) from the TS/PS packetizing module 41 , it restores 
an individual packetized elementary stream (PES) by 
combining or dividing packetized elementary streams 
(PES) contained in the transport stream (TS). It then ex- 
tracts a restored packetized elementary stream (PES). 
This is because each transport stream (TS) contains a 
part of a packetized elementary stream (PES) or a com- 
bination of a plurality of packetized elementary streams 
(PES). When the comparator 45 receives a program 
stream (PS) from the TS/PS packetizing module 41, it 
deletes the header from the program stream (PS) and 
extracts the packetized elementary stream (PES). 
Then, the comparator 45 compares the extracted pack- 
etized elementary stream (PES) with the non-multi- 
plexed packetized elementary stream (PES) stored in 
the FIFO memory 42 and sends the comparison result 
to the control module 5. 



The other two comparators 46 and 47 do the same 
processing. That is, the comparator extracts voice data 
or private data from a transport stream (TS) or a pro- 
gram stream (PS), compares the extracted data with the 
s non-multiplexed packetized elementary stream (PES) 
stored in the FIFO memory 43 or 44, and informs the 
control module 5 of the comparison result. 

Based on the comparison result sent from each of 
comparators 1 3, 23, 33, 45, 46, and 47, the control mod- 
's ule 5 checks if an error occurred in the PES packetizing 
modules 11, 21, 31 or in the TS/PS packetizing module 
41 and, based on the result, controls the operation of 
the packetizing module 11, 21, 31, or 41. For example, 
when there is no error in the packetizing module 11,21, 
'5 31, or 41, the control module 5 treats the output from 
the packetizing module 11, 21, 31, or 41 as valid; con- 
versely, if there is an error in the packetizing module 1 1 , 
21 , 31 , or 41 , the control module 5 treats the output as 
invalid. Or, the control module 5 may be designed so 
20 that it can tell the packetizing module to re-generate the 
transport stream (TS) or program stream (PS) when 
there is an error in the packetizing module 11 , 21 , 31 , 
or 41. 

As described above, this system has the FIFO 

25 memories 12, 22, and 32 to compare an non-packetized 
elementary stream (ES) with that a packetized elemen- 
tary stream (ES). The result indicates whether an error 
occurred in any of the PES packetizing modules. This 
system also has the FIFO memories 42, 43, and 44 to 

so compare a non-multiplexed packetized elementary 
stream PES with a multiplexed packetized elementary 
stream PES. The result indicates whether an error oc- 
curred in the TS/PS packetizing module 41. Compared 
with a system with a separate de-multiplexing module, 

35 this simply-configured system detects a data error in the 
payload field that may occur in the multiplexing module, 
increasing reliability. 

In addition, use of the FIFO memories eliminates 
the need for the comparators 1 3, 23, 33, 45, 46, and 47 

40 or the control module 5 to manage the location of an 
elementary stream (ES) or a packetized elementary 
stream (PES), reducing the processing load. 

This multiplexing module also has a FIFO memory 
and a comparator for each elementary stream (ES) and 

*s a packetized elementary stream (PES), thus enabling 
the control module to identify which packetizing module 
has an error. 

<Second Embodiment 

In the second embodiment, the present invention is 
applied to a multiplexing system which has a plurality of 
multiplexing modules and which selects the output from 
an error-free multiplexing module. As shown in Fig. 6, 
55 this multiplexing system has two sets with the configu- 
ration shown in Fig. 1 , each set including the multiplex- 
ing module 1 and the control module 5. This system also 
has the selection control module 52 to select one of the 
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multiplexing modules, depending upon the error output 
from the control modules 5a and 5b of the multiplexing 
modules 1a and 1b, 

The comparators 1 3, 23, 33, 45, 46, and 47 in each 
of the multiplexing modules 1 a and 1 b send to the con- 
trol modules 5a and 5b the result of comparison be- 
tween a non-packetized elementary stream (ES) and a 
packetized elementary stream (ES) as well as the result 
of comparison between a non-multiplexed packetized 
elementary stream (PES) and a multiplexed packetized 
elementary stream (PES). 

Each of the control modules 5a and 5b checks the 
comparison output from the comparators 1 3, 23, 33, 45, 
46, and 47 to check to see if there is a comparison error 
and, if it finds an error, informs the selection control mod- 
ule 52 of the error. Based on the information from the 
control modules 5a and 5b, the selection control module 
52 checks if an error occurred in the multiplexing mod- 
ules 1a and 1b. The selection control module 52 then 
controls the TS/PS packetizing module 41 in the multi- 
plexing modules 1a and 1b in such a way that a transport 
stream (TS) or a program stream (PS) from an error- 
free multiplexing module 1 a or 1 b is output to the com- 
munication line or some other unit. 

For example, an error in one of the packetizing mod- 
ules 11, 21, 31, and 41 shows that a packetizing error 
or a multiplexing error occurred and that the non-pack- 
etized elementary stream (ES) or packetized elementa- 
ry stream (PES) and the packetized elementary stream 
(ES) or packetized elementary stream (PES) differ. 
When one of the comparators 1 3, 23, 33, 45, 46, and 
47 finds this difference, the control module 5a or 5b finds 
an error, causing the selection control module 52 to se- 
lect an error-free multiplexing module 1a or 1b. There- 
fore, this multiplexing system is able to output an error- 
free transport stream (TS) or a program stream (PS), 
increasing the reliability of the multiplexing module. 

Another advantage of this system is that, when an 
error occurs in both multiplexing modules 1 a and 1b, the 
control modules 5a and 5b detect the number of errors 
in each comparatorto allow the selection control module 
52 to send the output sent from the multiplexing module 
1 a or 1 b which has less errors. This prevents a transport 
stream (TS) or a program stream (PS) from being dis- 
rupted even when an error occurs in both multiplexing 
module 1a and 1b. 

In the above embodiments, a FIFO memory is used 
to store packets to be multiplexed. This memory may be 
replaced, as necessary, by another type of memory. 

The packet multiplexing system according to the 
present invention compares the original input data with 
the input data in the packet using the comparison means 
to check whether those input data are similar or not. Ac- 
cordingly, the packet multiplexing system can detect er- 
rors lying therebetween without the packet de-multiplex- 
ing module serving as the monitor circuit for the input 
data in the multiplexing system, unlike the conventional 
arts. As described above, the de-multiplexing module is 



so large and expensive; a compact and low-cost packet 
multiplexing system can be available. 

Furthermore, the packet multiplexing system syn- 
chronizes the original input data with the input data in 
s the packet using the synchronization means, which ad- 
justs output timing of the original input data. Therefore, 
the packet multiplexing system can detect difference be- 
tween the original input data and the input data in the 
packet based on those data synchronized with each oth- 

Moreover, the packet multiplexing system adjust 
the output timing of the original input data by a prede- 
termined delay time or a time necessary for packetizing 
of the original input data, for the purpose of the synchro- 
's nization. As a result, the packet multiplexing system can 
accurately confirm the difference between the original 
input data and the input data in the packet. 

For an application of the packet multiplexing sys- 
tem, the packet multiplexing equipment is constructed 
so of a plurality of the packet multiplexing system. The mul- 
tiplexing equipment detects any faults of one among 
those packet multiplexing systems, thus selecting one 
of the others capable of regular operation. Consequent- 
ly, even though one of those packet multiplexing sys- 
25 terns malfunctioned, the multiplexing equipment can 
continue the multiplexing of the data using another 
packet multiplexing system free of the fault, thereby 
leading to enhancement of reliability of the multiplexing. 



Claims 

1. A packet multiplexing system comprising: 

35 packet generation means for generating from 

input data a packet containing the input data 
and a header; and 

comparison means for comparing the input da- 
ta with the input data in the packet to check dif- 
*o ference therebetween. 

2. A packet multiplexing system as set forth in claim 1 
further comprising: 

4S synchronization means for synchronizing the 

input data with the input data in the packet, 
wherein the comparison means compares the 
synchronized input data with the input data in 
the packet to check difference therebetween. 

3. A packet multiplexing system as set forth in claim 2 
wherein the synchronization means has delay 
means for allowing the input data to be output with 

4. A packet multiplexing system as set forth in claim 
3, wherein the packet generation means has mon- 
itor means for monitoring generation of the packet 
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and for outputting a signal indicating the completion 
of the generation of the packet and size of the head- 
er, and the delay means allows the input data to be 
output according to the signal. 

5 

5. A packet multiplexing system as set forth in claim 
2, wherein the synchronization means has output 
adjustment means for outputting the input data up- 
on expiration of sum of a first time for the packet 
generation means to generate the packet and a io 
second time corresponding to header of the packet 
from receipt time of the input data. 

6. A packet multiplexing system comprising: 



chronizing the input data with the input data in the 
first packet, and 

the second packet multiplexing system further com- 
prises second synchronization means for synchro- 
nizing the input data with the input data in the sec- 
ond packet and wherein 

the first comparison means compares the syn- 
chronized input data with the input data in the 
first packet and 

the second comparison means compares the 
synchronized input data with the input data in 
the second packet. 



first packet generation means for generating 
from input data a first packet containing the in- 
put data and a first header; 
second packet generation means for generat- 
ing from the first packet a second packet con- 20 
taining the first packet and a second packet 
:ont ,,t, ng a second header: 
first comparison means for comparing the input 
data with the input data in the first packet to 
chock difference therebetween; and 2s 
second comparison means for comparing the 
first packet with the first packet in the second 
packet to check difference therebetween. 



7. A packet multiplexing system comprising a first 30 
packet multiplexing system and a second packet 
multiplexing system wherein the first packet multi- 
plexing system comprising: 



first packet generation means for generating 35 
from input data a first packet containing the in- 
put data and a header; and 
first comparison means for comparing the input 
data with the input data in the first packet to 
check difference therebetween; and wherein, 40 
the second packet multiplexing system com- 
prising: 

second packet generation means for generat- 
ing from the input data a second packet con- 
taining the input data and a second packet con- *5 
taining the input data and the header; and 
second comparison means for comparing the 
input data with the input data in the second 
packet to check difference therebetween, and 
the first packet multiplexing system and the so 
second packet multiplexing system are used 
selectively based on comparison results of the 
first comparison means and the second com- 
parison means. 



8. A packet multiplexing system as set forth in claim 
7, wherein the first packet multiplexing system fur- 
ther comprises first synchronization means for syn- 
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